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Abstract. It has now become an important issue to evaluate the performance of technical institu¬ 
tions to develop better research and enrich the existing teaching processes. The results of such per¬ 
formance appraisal would serve as a reference point for decisions to choose a particular institution, 
hire manpower, and provide financial support for the betterment of students and underperforming 
institutions. The performance of institutions responsible for promotion of technical education in 
a vast country, like India also needs to be assessed for its journey to share a major role in global 
economy. In this paper, an integrated approach combining PROMETHEE (Preference Ranking 
Organization METHod for Enrichment Evaluation) and GAIA (Geometrical Analysis for Interac¬ 
tive Aid) methods is applied for evaluating the performance of 20 National Institutes of Technol¬ 
ogy (NITs) in India with respect to nine pivotal criteria. It is observed that NIT, Tiruchirappalli 
and NIT, Warangal are the two best performers based on the last five years data. On the other 
hand, NIT, Patna takes the last position in the ranking list of 20 NITs suggesting allocation of more 
resources for its efficient management and subsequent development. Placement of the graduated 
students from those NITs is found to be the most important criterion. 

Keywords: technical institution; NIT, PROMETHEE, GAIA, Shannon’s entropy method. 


1. Introduction 

In the age of mechanics, technical education is essential to run our factories and fields 
of production. The need for technical education in India is no longer a subject of debate 
as it plays a vital role in our national and global development. Indian technical educa¬ 
tion system is one of the largest educational systems in the world. After independence in 
1947, based on the recommendations of the Planning Commission in 1955 to promote 
technical education in India and develop India as a leader in science and technology, 14 
Regional Engineering Colleges (RECs) were first established between 1959 and 1965. 
The main aim behind setting up those technical institutions was to provide the required 
technical manpower for the industrial projects being contemplated during the 2 nd Five- 
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Year Plan (1956-1961). Later, three other new RECs were established. In 2003, all RECs 
became National Institutes of Technology (NITs). In addition, the Central Government 
of India had taken over three other institutes and converted them into NITs. In 2010, the 
government had announced setting up ten more new NITs in the remaining states/union 
territories, leading to establishment of NITs in every state in India. It means that there 
are now altogether 30 NITs in India. 

As the uniform quality output has become the prime concern today, therefore, per¬ 
formance evaluation and ranking of these NITs have now been a key research issue. The 
mechanism used to evaluate NITs plays an important role in guiding the development of 
and the government’s financial support for each NIT. The evaluation results would serve 
as a reference point for decisions to choose a NIT, employ suitable resources, identify 
underperforming NITs and provide direction for subsequent improvement. It would also 
help the NITs develop better research and teaching processes. As all the stakeholders 
(students as well as researchers) want to get the optimal benefits at the shortest period of 
time and at an economical cost to have quality education, this is the high time to analyze 
and evaluate the performance of these NITs. With this aim, this paper focuses on the ap¬ 
plication of integrated PROMETHEE (Preference Ranking Organization METHod for 
Enrichment Evaluation) and GAIA (Geometrical Analysis for Interactive Aid) methods 
for assessing the performance of 20 NITs in India with respect to nine evaluation criteria. 
The weights of the considered criteria are determined using Shannon’s entropy method. 
Here, PROMETHEE II method provides the total ranking of all the 20 considered NITs, 
and GAIA method acts as a graphical decision aid to identify the best NIT along with its 
positive dimensions and show the effects of changing criteria weights on the final rank¬ 
ing of these NITs. It also searches out the worst performing NIT along with its major 
weaknesses where special emphasize needs to be provided. 


2. Review of the Past Researches 

Johnes and Johnes (1995) applied data envelopment analysis (DEA) to investigate the 
technical efficiency of UK university departments of Economics as producers of research, 
and concluded that DEA had a positive contribution to make in the development of mean¬ 
ingful indicators of university performance. Johnes (1996) examined the possibility of 
constructing measures of performance of UK universities, and developed a methodology 
in the framework of production theory using multiple regression techniques to estimate 
the relationship between the outputs and inputs of the universities. Athanassopoulos and 
Shale (1997) examined the comparative efficiency of higher education institutions in the 
UK using DEA model. The application of DEA model to 45 universities in the UK short¬ 
listed a subset of six institutions showing satisfactory performance across alternative ef¬ 
ficiency tests. McMillan and Datta (1998) reported the results of using DEA model to as¬ 
sess the relative efficiency of 45 Canadian universities. Korhonen et a\. (2001) proposed 
a systematic approach to analyze academic research performance at universities and 
research institutes. It was concluded that the developed approach would enable a univer¬ 
sity to allocate resources more efficiently to its research units. Avkiran (2001) used DEA 
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method to examine the relative efficiency of Australian universities, and concluded that 
the university sector was performing well on technical and scale efficiency, but there 
was scope for improving performance on fee-paying enrolments. Moreno and Tadepalli 
(2002) used DEA technique for evaluating the efficiency of academic departments at a 
public university. Using DEA method, Lopes and Lanzer (2002) evaluated the perfor¬ 
mance of 58 departments in a Brazilian university and showed that 15 of those depart¬ 
ments had low performance. Abbott and Doucouliagos (2003) applied non-parametric 
techniques to estimate technical and scale efficiency of individual Australian universi¬ 
ties. Flegg et al. (2004) used DEA method to examine the technical efficiency of 45 Brit¬ 
ish universities in the period 1980/81—1992/93. Casu and Thanassoulis (2006) applied 
DEA model to identify practices leading to cost-efficient central administrative services 
in UK universities. Reichmann and Sommersguter-Reichmann (2006) proposed a frame¬ 
work for assessing the technical efficiency of 118 university libraries from Australia, 
Austria, Canada, Germany, Switzerland and the United States. The DEA model was 
chosen to analyze the performance differentials of university libraries. Johnes (2006) ap¬ 
plied DEA method to 2547 Economics graduates from UK universities in 1993 in order 
to assess teaching efficiency. Fandel (2007) presented a solution for redistributing funds 
for teaching and research among the universities in North Rhine-Westphalia in Germany 
according to some specific criteria. Kao and Hung (2008) applied DEA method to assess 
the relative efficiency of the academic departments at National Cheng Kung University 
in Taiwan, considering total credit hours, publications and external grants as outputs, 
and number of personnel, operating expenses and floor space as inputs. Worthington and 
Lee (2008) investigated the productivity growth in 35 Australian universities using non- 
parametric frontier techniques, taking into account full time equivalent academic and 
non-academic staff, non-labor expenditure, and undergraduate (UG) and postgraduate 
(PG) student load as the inputs, and UG, PG and Ph.D. completions, national competi¬ 
tive, industry grants and publications as the outputs. Tyagi et al. (2009) evaluated the 
performance efficiencies of 19 academic departments of Indian Institute of Technology 
(IIT), Roorkee through DEA technique. Chen and Chen (2010) developed an innovation 
support system for Taiwanese higher education institutes to comprehensively evaluate 
their innovation performance. The decision-making trial and evaluation laboratory (DE- 
MATEL), fuzzy analytic network process (FANP) and technique for order preference by 
similarity to an ideal solution (TOPSIS) were utilized considering the interdependence 
and relative weight of each measurement criterion. Giannoulis and lshizaka (2010) de¬ 
veloped a three-tier Web-system for customized ranking of British universities using 
ELECTRE III (ELimination and Et Choice Translating REality) method. Lee (2010) 
developed an intellectual capital evaluation model as a university assessment scheme 
in Taiwan. Pouris and Pouris (2010) identified the international standing of the South 
African universities in various scientific disciplines, compared them with their standing 
over time and focused on the consequences relevant to higher education policy. Kuah 
and Wong (2011) presented a DEA model for jointly evaluating the relative teaching and 
research efficiencies of universities. Abramo et al. (2011) proposed an application of 
DEA methodology for measurement of technical and allocative efficiency of university 
research activity. Agha et al. (2011) applied DEA model to assess the relative technical 
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efficiency of the academic departments, considering operating expenses, credit hours and 
training resources as the inputs, and number of graduates, promotions and public service 
activities as the outputs. Li (2011) used DEA method to analyze and evaluate the output 
efficiency, scale efficiency and deficiency of 42 universities’ human resources. Wu et al. 
(2012) determined the performance evaluation indices for higher education for ranking 
12 private universities in Taiwan. A combined approach using analytic hierarchy process 
(AHP) and VIKOR (VlseKriterijumska Optimizacija I Kompromisno Resenje) method 
was proposed. Das et al. (2012) focused on the performance evaluation and ranking 
of seven IITs in respect to stakeholders’ preference using an integrated model of fuzzy 
AHP and COPRAS (COmplex PRoportional Assessment). It was observed that the per¬ 
formance of two IITs would need considerable improvement. Hashemkhani Zolfani and 
Safaei Ghadikolaei (2013) applied DEMATEL for research on cause and effect relations 
of perspectives of balanced scorecard, ANP to calculate weights of indices in perspec¬ 
tives and VIKOR for ranking five universities in Iran. In Table 1, a concise representa¬ 
tion of the performance evaluation methodologies adopted, sample where the evaluation 
was performed and criteria/sub-criteria as considered by the past researchers is provided. 
It is found that most of the past researchers adopted DEA methodology for assessing the 
performance and ranking of various universities/institutions/libraries/research centers. 
But the mathematical models as required for deployment of DEA technique are too com¬ 
plex to comprehend and it is able to shortlist only the efficient decision-making units/ 
alternatives. When a complete ranking preorder of the considered alternatives/options 
is desired, it fails to fulfill this requirement. Lengthy linear programming formulations 
often need to develop for its solution. For this reason, an attempt is made in this paper to 
augment a simple integrated approach combining PROMETHEE and GAIA methods to 
evaluate the performance of 20 NITs in India. 


Table 1 

Methodology, sample and criteria/sub-criteria considered by the past researchers 


Author 

Sample 

Methodology 

adopted 

Criteria/Sub-criteria 

Johnes and 

Johnes(1995) 

36 Economics 
departments in 
UK university 

DEA 

Teaching/research staff, per capital research grant, UG 
student load, papers and letters in academic journals, 
articles in professional and popular journals, authored 
and edited books, published official reports, contributions 
to edited works 

Johnes(1996) 

41 British 
universities 

Multiple 

regression 

technique 

Employee, degree, non-completion, research 

Athanassopoulos 
and Shale (1997) 

45 universities 
in UK 

DEA 

No. of PGs, No. of UGs, No. of academic staff, mean 
A-level entry score, research income, expenditure 
on library and computing services, No. of successful 
leavers, No. of higher degrees awarded, weighted 
research rating 

McMillan and 
Datta (1998) 

45 Canadian 
universities 

DEA 

No. of faculties, total expenditure, UG teaching, gradu¬ 
ate teaching 

Continued in next page 
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Author 

Sample 

Methodology 

adopted 

Criteria/Sub-criteria 

Korhonen et al. 
(2001) 

18 research 
units in Finland 

DEA 

Quality of research, research activity, impact of re¬ 
search, activity in educating young scientists, activity 
in the scientific community 

Avkiran (2001) 

36 Australian 
universities 

DEA 

Overall performance, performance on delivery of edu¬ 
cation services, performance on fee-paying enrolments, 
academic staff, non-academic staff 

Moreno and 
Tadepalli (2002) 

42 departments 
in a public 
university 

DEA 

Faculty and staff salary, operational and equipment 
budget, allocated space, No. of UGs, full time equiva¬ 
lents, student credit hours, amount of grants awarded 

Lopes and Lanzer 
(2002) 

58 departments 
in a Brazilian 
university 

DEA 

Faculty salary, staff salary, operational budget, 
equipment budget, allocated space, No. of UGs, full 
time equivalents, student credit hours, grants awarded 

Abbott and 

Doucouliagos 

(2003) 

36 Australian 
universities 

DEA 

No. of academic staff, No. of non-academic staff, expen¬ 
diture of all non-labor inputs, value of non-current assets, 
research quantum, No. of equivalent full time students 

Flegg et al. 

(2004) 

45 UK 
universities 

DEA 

No. of staff, No. of UG students, No. of PG students, 
aggregate departmental expenditure, income from re¬ 
search and consultancy, No. of UG degrees awarded, 
No. of PG degrees awarded 

Casu and 
Thanassoulis 
(2006) 

108 universities 
in UK 

DEA 

Total administrative cost, total income from students, 
total staff cost, technology transfer 

Reichmann and 
Sommersguter- 
Reichmann 
(2006) 

118 university 
libraries 

DEA 

Total current library expenditure, current labor cost, 
No. of full time equivalents, No. of book materials 
held, total area 

Johnes(2006) 

2547 Economics 
graduates from 
UK universities 

DEA 

A-level score, gender, school type, nationality, No. of 
graduates 

Fandel (2007) 

15 German 
universities 

DEA 

No. of academic personnel, No. of students, third-party 
(outside) funding, No. of graduates, No. of doctorates 
in each university 

Kao and Hung 
(2008) 

41 departments 
in a Taiwan 
university 

DEA 

No. of personnel, expenses, space, teaching credit 
hours, publications, grants 

Worthington and 
Lee (2008) 

35 Australian 
universities 

DEA 

Academic staff, non-academic staff, non-labor expen¬ 
diture, UG student load, PG student load, No. of 
awards, No. of publications, research income 

Tyagi et al. 

(2009) 

19 departments 
in I IT, Roorkee 

DEA 

Academic staff, non-academic staff, departmental ope¬ 
rating cost, total enrolled student, progress, research 
index 

Chen and Chen 
(2010) 

Taiwanese 
higher education 
institutes 

DEMATEL, 
FANP and 
TOPSIS 

Research patents, international academic interaction, 
No. of R & D members, financial support of national 
science council, journal papers accepted and published, 
operation electrification, outsourcing, affair rotation, 
information study camp, faculty and staff, government 
tender planning, refresher class, go abroad for further 
education, No. of conferences, No. of international 
students in council, No. of chair professors, organizational 
structure, organizational culture 




Continued in next page 
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Author 

Sample 

Methodology 

adopted 

Criteria/Sub-criteria 

Giannoulis and 
Ishizaka (2010) 

113 British 
universities 

ELECTRE III 

Student satisfaction, research assessment, entry stan¬ 
dards, staff/student ratio, academic services spend, 
facilities spend, good honours, graduate prospects, 
completion 

Lee (2010) 

A Taiwanese 
university 

Fuzzy AE1P 

Administration, curriculum, technology transfer, re¬ 
search, teaching, service 

Pouris and Pouris 
(2010) 

7 South African 
universities 

A ranking ap¬ 
proach based 
on citations 

Essential science indicators 

Kuah and Wong 
(2011) 

30 universities 

DEA 

No. of academic staffs and taught course students, ave¬ 
rage student’s qualifications, university expenditures, 
No. of research staffs and research students, average 
research staff’s qualifications, research grants 

Abramo et al. 
(2011) 

28 Italian 
universities 

DEA 

Staff-years of full professors, staff-years of associate 
professors, staff-years of assistant professors, sci¬ 
entific strength 

Agha et al. 

(2011) 

30 departments 
in a university 
in Gaza 

DEA 

Operating expenses, load in hours, training resources, 
No. of graduates, promotions, public service activities 

Li (2011) 

42 universities 

DEA 

Floor area, library collection size, full time teachers, sci¬ 
entific research expenditure, discipline level, students 
scale, monthly salary of students after 6 months from 
graduation, ability to work 

Wu et al. (2012) 

12 private 
universities in 
Taiwan 

AHP and 
VIKOR 

Fund and resources, No. of journals, size of school area, 
foreign faculties and students, courses taught in English, 
international activities, income of extension education, 
investment of extension education, student club and pail 
time job, student loan, employment consulting, credits 
of general education course, manpower administrative 
support, No. of research rooms, full and part tune 
faculty, teacher-student ratio, ratio of foreign faculty 
classes, amount of course credits, status of students, tea¬ 
ching hours of faculty, publication of books and papers, 
international journal and conference papers, national 
science council projects, journal reviewers, ratio of 
doctoral students 

Das et al. (2012) 

7 IITs in India 

Fuzzy AE1P 
and COPRAS 

Faculty strength, student intake, No. of Ph.D. awarded, 
No. of patents applied for, campus area, tuition fee 

Hashemkhani 
Zolfani and Safaei 
Ghadikolaei 
(2013) 

5 universities 
in Iran 

DEMATEL, 
ANP and 
VIKOR 

Financial, customer, internal process, learning and 
growth 


3. PROMETHEE Method 


The PROMETHEE (Preference Ranking Organization METHod for Enrichment Evalu¬ 
ation) is a multi-criteria decision-making (MCDM) method developed by Brans et al. 
(Brans and Vincke, 1985; Brans et al, 1986; Brans and Mareschal, 1994). It is a simple 
ranking method in conception and application compared to other MCDM techniques. 
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The PROMETHEE method can be applied to various decision-making situations 
where the best alternative/option needs to be chosen from a finite set of feasible alter¬ 
natives based on several conflicting criteria. An excellent review on the application of 
PROMETHEE method in various domains of engineering and management can be avail¬ 
able in (Behzadian et al, 2010). The PROMETHEE method starts with the following 
decision (evaluation) matrix: 


gl(fll) 

g2 Ol) 

- g/(«i) 

- g„(a 1 ) 

gl(tf 2 ) 

g 2 (a 2 ) 

... gj{a 2 ) 

- g„(« 2 ) 

g\(. a i) 

& 2 (a,') 

- g/K) 

- g„{a t ) 

_£lOm) 

giiaj 

■ ■ ■ gji ) 

••• gn(^m) 


where g-( a,) shows the performance of z lh alternative on j th criterion, 
m is the number of alternatives and n is the number of criteria. 

Implementation of PROMETHEE method requires two additional information, i.e. 
a) relative importance or weights/priorities of the considered criteria, and b) the deci¬ 
sion maker’s preference function for comparing the contribution of the alternatives with 
respect to each criterion. The AHP or Shannon’s entropy method can be applied for 
determination of the criteria weights. The preference structure of PROMETHEE method 
is based on pair-wise comparisons. In this case, the deviation between the evaluations 
of two alternatives on a particular criterion is considered. For small deviation, the deci¬ 
sion maker would allocate a small preference to the best alternative and even possibly 
no preference if the decision maker considers that this deviation is negligible. The larger 
the deviation, the larger is the preference. These preferences are real numbers varying 
between 0 and 1. This means that for each criterion, the decision maker considers the 
following preference function: 

Pj(a, b ) =Fj[dj(a,b)] Va,b g A (2) 

where dj(a, b) = [g y (zz) - g y (b)] and 0 <Pj{a, b) < 1. 

For beneficial criteria, this function gives the preference of ‘o’ over ‘ b’ for the ob¬ 
served deviations between their evaluations on criterion gj(.). The preference equals 
to 0 when the deviations are negative. 

For non-beneficial criteria, the preference function should be reversed as follows: 

P j (a,b)=F ] [-dj(a,b)\ (3) 

A generalized criterion, (g y (.), P, (a, b)} is associated to criterion g, (.) and it is 
to be defined for each criterion. In order to facilitate selection of a specific preference 
function, six basic types of preference function, i.e. usual criterion, U-shape criterion, 
V-shape criterion, level shape criterion, V-shape with indifference criterion and Gauss- 
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ian criterion are proposed to the decision maker. In some of these preference functions, 
different threshold parameters ( p, q or s) need to be set by the decision maker. As the 
shapes of preference functions and their parameters are chosen by the decision maker, 
it exerts clear advantages and features of PROMETHEE method (Podvezko and Pod- 
viezko, 2010). 

As soon as the evaluation matrix, g, (.) is developed, and the relative importance 
(weight), w .and generalized criterion, [g,(.). P f (a.b)\ are defined, PROMETHEE meth¬ 
od becomes ready for implementation. PROMETHEE method is based on pair-wise 
comparisons where the aggregated preference indices are defined as follows (Brans and 
Vincke, 1985): 


n(a,b) = ^ 'Pj(a,b)Wj 

J n ( 4 ) 

n(b,ci) = ^P j (b,a)w j 

7=1 

where x(a,b) expresses the degree with which ‘a’ is preferred to ‘V over all the criteria 
and n ( b , a) represents how ‘b’ is preferred to ‘a’. In most of the cases, there are criteria 
for which ‘a’ is better than ‘b\ and criteria for which ‘ b ’ is better than ‘o', and conse¬ 
quently, x(a,b) and x ( b, a ) are usually positive. 

As soon as the values of x{a, b ) and x(b,a) are computed for each pair of alterna¬ 
tives in the evaluation matrix, a complete outranking graph, including two arcs between 
each pair of nodes (alternatives), can be obtained. In this method, each alternative ‘a’ 
faces (m— 1) number of other alternatives in the evaluation matrix. Now, the following 
two outranking flows can be defined: 

Positive outranking flow, 

<P\a) = — Y/r(a,x) (5) 

m — 1 


Negative outranking flow, 


<P (a) 


1 


m — \ 


^~V(x,a) 

xeA 


( 6 ) 


The positive outranking flow expresses how an alternative ‘a’ outranks all other al¬ 
ternatives. The higher the value of <p + (a), the better is the alternative. The negative out¬ 
ranking flow expresses how an alternative ‘o’ is being outranked by the others. Lower 
value of <p~(a) signifies better alternative. In PROMETHEE I method, the partial ranking 
(P\ I 1 , R 1 ) is obtained from the positive and negative outranking flow values where P\ 
P and R l respectively stand for preference, indifference and incomparability relations. 
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Both the flows do not usually induce the same rankings. PROMETHEE II method can 
provide a complete preorder of the alternatives by using a net flow, though it loses much 
information of preference relations. In this method, there is a balance between the posi¬ 
tive and negative outranking flows. The net outranking flow for each alternative can be 
obtained using the following equation: 

(p{a) = cp + {a)-(p-{a) (7) 

The higher the value of tp(a), the better is the alternative. Thus, the best alternative is 
the one having the highest tp(a) value. PROMETHEE II provides a complete ranking of 
the alternatives from the best to the worst one. Since PROMETHEE I does not provide 
a complete ranking, the resulting ranking cannot be compared with the ranking provided 
by PROMETHEE II method. PROMETHEE I ensures creation of indifferent and incom¬ 
parable alternatives. Other versions of PROMETHEE method, such as PROMETHEE III 
for ranking based on interval, PROMETHEE IV for complete or partial ranking of the al¬ 
ternatives when the set of viable solutions is continuous, PROMETHEE V for problems 
with segmentation constraints, PROMETHEE VI for human brain representation, PRO¬ 
METHEE GDSS for group decision-making and the visual interactive module GAIA 
for graphical representation are also available to help the decision makers to solve more 
complicated decision-making problems (Behzadian et al., 2010). 


4. GAIA Method 

From the positive and negative outranking flow values, as expressed in Eqns. (5) and (6) 
respectively, the net outranking flow value can be estimated, as given below: 

1 " 

<P(a) = <P + (a)-<p~(a) = - -Y Y [P(a,x) - P (x,a)]w■ (8) 

m- 1 , 

7=1 xeA 

Consequently, 

n 

<p{a)=Y J ( PM)w j (9) 

j =i 

if 

<p {a) = —- Y [Pj(a,x) - Pj(x,a)] (10) 

m -1 ^7 

xeA 


(Pj{a) is the single criterion net flow obtained when only criterion g .(.) is considered. 
It expresses how an alternative 'a' is outranking (< p.(a ) > 0) or outranked {(p <( a )< 0) by 
all other alternatives on criterion g ; (.). From Eqn. (9), it is observed that the global net 
flow of an alternative is the scalar product between the vector of criteria weights and the 
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profile vector of that alternative. This property is extremely useful while developing the 
GAIA plane. Now, the following matrix M (m x n) can be defined based on the single 
criterion net flows of all the alternatives. 



<P2(ci\) 


• <P„(ai ) 

<P\(a 2 ) 

<P 2 ( a 2) ■ 

■ <Pj( a 2) 

■ <P„(a 2 ) 

< P\( a i) 


■ f Pj( a i) 

• %(«,) 

VMm) 


■ <Pj(a m ) • 



The information included in matrix M is more extensive than the one in the decision/ 
evaluation matrix, as given in Eqn. (1), because the degrees of preference given by the 
generalized criteria are taken into consideration in M. Moreover, the g J (a j ) values are 
expressed on their own scale, while the <pj(a ,) values are dimensionless. In addition, the 
matrix M is not dependant on the weights of the considered criteria. 

Consequently, the set of m alternatives can be represented as a cloud of in points 
in a ^-dimensional space. As the number of criteria is usually larger than two, it is im¬ 
possible to obtain a clear view of the relative positions of the points with regard to the 
criteria. The GAIA plane is obtained by projection of this on a plane such that as few 
information as possible get lost. In this plane, alternatives ( a u a 2 ,... a m ) are represented 
by points and the criteria (c b c 2 , ...,c„) by axes, as shown in Fig. 1. From Eqn. (9), it can 
be said that the PROMETHEE net flow of a, is the projection of the vector of its single 
criterion net flows on w. Consequently, the relative positions of the projections of all 
the alternatives on w provides the PROMETHEE ranking of the alternatives. Clearly, 
the vector w plays a pivotal role. It is represented in GAIA plane by the projection of 



Fig. 1. A GAIA plane. 
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the unit vector of the weights. This projection is referred to as the PROMETHEE deci¬ 
sion axis ( n ). This axis shows the direction of the compromise solution resulting from 
the weights allocated to the criteria. If the weights are concentrated on one criterion, 
it is clear that the n axis will coincide with the axis of that criterion in GAIA plane. 
When the weights are distributed over all the criteria, the n axis appears as a weighted 
resultant of all the criteria axes. 

If 7r is long, the PROMETHEE decision axis has a strong decision power and the 
decision maker is invited to select alternatives as far as possible on its direction. On 
the other hand, when n is short, it has no strong decision power, which means that 
according to the weights, the criteria are strongly conflicting and the selection of a 
good compromise solution is a hard problem. In this plane, criteria expressing similar 
preferences on the evaluation data are oriented in the same direction, and the conflict¬ 
ing criteria are pointing in the opposite directions. When the weights are modified, the 
positions of the alternatives and of the criteria remain unchanged in GAIA plane. The 
weight vector appears as a decision stick that the decision maker can move accord¬ 
ing to the preference in favor of a particular criterion. When a sensitivity analysis is 
performed by modifying the weights, the PROMETHEE decision stick (w) and the n 
axis are moving in such a way that the consequences for decision-making are easily 
observed in GAIA plane. 

5. Shannon’s Entropy Method 

Although there are several distinct methods to determine weights/relative importance of 
criteria as involved in a decision-making problem, like AHP, Delphi method and Shan¬ 
non’s information entropy method, the latter one is usually adopted as it can avoid the 
effect of subjective judgments while pair-wise comparing the relative importance of the 
considered criteria (Shanon, 1948). Entropy is a measure that uses probability theory to 
measure the uncertainty of information. It shows that the more dispersive the data, the 
bigger the uncertainty. If there is a decision matrix, A consisting of m number of alterna¬ 
tives and n number of criteria, then the ratio of index of / lh criterion in I th alternative can 
be calculated as follows: 



(J= 1,2,..,«) 


( 12 ) 


where Xj is the performance of i lh alternative with respect to j th criterion. 

Now, the decision information of each index can be expressed by the entropy value 
( Cj ) using the following equation: 




(13) 
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The Shannon’s entropy weight for j th criterion can now be obtained employing the 
following expression: 


1 ~ e J 

£"=tO - e j ) 


(14) 


Entropy method has several advantages over the others, i.e. it can be applied in case 
of fuzzy data, it can be adopted for problems with high complexity involving a large 
number of criteria, and it can be modified and used for non-deterministic data, such as 
interval data while producing interval weights. 


6. NITs in India 

After independence, Jawaharlal Nehru sought to develop India as a leader in science 
and technology. On the recommendations of Engineering Personnel Committee (EPC) 
set up by the Planning Commission in 1955, the Government of India started 14 RECs 
between 1959 and 1965 at Bhopal (in 1960), Allahabad (in 1961), Kozhikode (Calicut) 
(in 1961), Durgapur (in 1960), Kurukshetra (in 1963), Jamshedpur (in 1960), Jaipur (in 
1963), Nagpur (in 1960), Rourkela (in 1961), Srinagar (in 1960), Surathkal (in 1960), 
Surat (in 1961), Tiruchirappalli (in 1964) and Warangal (in 1959). Those RECs were set 
up as joint and co-operative ventures of the Central and State Governments concerned 
with a view to create the required technical manpower by providing undergraduate edu¬ 
cation and training in different branches of engineering and technology for the industrial 
projects being contemplated during the 2 nd Five-Year Plan (1956-1961). Later, three 
other new RECs in Silchar (in 1967), Hamirpur (in 1986) and Jalandhar (in 1987) were 
established. Those institutes were registered as autonomous bodies under the Society 
Registration Act, 1860 and affiliated to the State Universities in their respective regions. 
The success of technology-based industry led to high demand for more technical and 
scientific education. Further, the RECs were also envisaged to function as pace setters 
and to provide academic leadership to the technical institutions. RECs were basically 
founded to promote regional diversity and multi-cultural understanding in India. 

Due to enormous cost and infrastructure involved in creating globally respected 
IITs, in 2002, Ministry of Human Resources and Development (MHRD) decided to 
upgrade RECs to NITs instead of creating new IITs. The Central Government controls 
those NITs and provides all funding. Thus, the NITs became a group of public engi¬ 
neering institutes of India. In 2003, all RECs became NITs. The upgrade was designed 
along the lines of the prestigious IITs after it was concluded that RECs had potential as 
proven by the success of their alumni and their contributions in the field of technical 
education. Subsequently, funding and autonomy for NITs increased, and those were 
taken over as fully funded institutes of the Central Government of India and granted 
deemed university status. In addition, the Central Government had also taken over three 
other institutes, i.e. Bihar Engineering College, Patna; Government Engineering Col- 
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lege, Raipur and Tripura Engineering College, Agartala, and converted them into NITs 
in 2004, 2005 and 2006 respectively. Thus, the total number of NITs had gone up to 20. 
Those institutes were expected to be at par with other national level technical institutes 
and be able to fulfill the demand of high quality undergraduate and postgraduate level 
of education in engineering and technology. An Act, i.e. the NIT Act, 2007 has since 
been enacted by the Parliament of India so as to provide a common statutory frame¬ 
work for all NITs and the Indian government has declared those schools as Institutes of 
National Importance. 

In 2010, the Government of India announced setting up ten new NITs in the remain¬ 
ing states/union territories, i.e. Delhi, Goa, Puducherry, Uttarakhand, Mizozam, Megha¬ 
laya, Manipur, Nagaland, Arunachal Pradesh and Sikkim. Comprising 30 autonomous 
institutes, this would lead to every state in India having its own NIT. Right from the 
beginning, NITs in Delhi, Goa, Puducherry and Uttarakhand share their place with the 
top NITs leaving behind old and private universities. Apart from their geographical ad¬ 
vantages, they also have top mentoring institutes. 

NITs offer degree courses at bachelors, masters and doctorate levels in various 
branches of engineering and technology. All NITs are autonomous which enables them 
to set up their own curriculum. Average NIT funding increased to Rs. 100 crores by 
year 2011. On average, each NIT also receives Rs. 20-25 crores under the World Bank 
funded Technical Education Quality Improvement Program (TEQIP). 

All NITs have a central library equipped with technical books, literature, fiction, sci¬ 
entific journals and other electronic materials. Most have digitized their libraries. Some 
provide an intranet library facility. Every departmental library has high speed connec¬ 
tivity. Electronic libraries allow students access to online journals and other periodicals 
through the AICTE-INDEST consortium. While some have video conferencing facili¬ 
ties, others are upgrading under the World Bank funded TEQIP scheme. 


7. Ranking of NITs Using PROMETHEE-GAIA Method 

It has already been observed that MCDM methods have immense potential in perfor¬ 
mance evaluation of competing alternatives (Zavadskas and Turskis, 2011; Zavadskas 
et al., 2014). Although, at present, there are 30 NITs in India, i.e. each state/union terri¬ 
tory has its own NIT, in this paper, the performance of 20 NITs is evaluated using PRO¬ 
METHEE-GAIA approach. Those ten NITs which were established during the year 
2009-2010 are not considered here due to non-availability of relevant data. From those 
NITs, students would be awarded with their UG/PG degrees only in year 2013-2014. 
So, some of the information as necessary for this evaluation process would not be 
available from those newly started NITs. The performance appraisal of the remaining 
20 NITs is carried out based on nine evaluation criteria, as enlisted in Table 2. All these 
nine criteria are so selected that they are entirely uncorrelated. Faculty strength is the 
total number of teaching faculties, including professors, associate professors and assis¬ 
tant professors in each NIT. Teacher student ratio is the ratio of the number of teaching 
faculties to the total number of enrolled students (UG, PG and doctoral students). The 
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Table 2 

Selected criteria for performance appraisal of 20 NITs 


Criteria Symbol 


Faculty strength Cj 

Teacher student ratio C 2 

Number of conferences held in last 5 years C 3 

Number of papers published in Journals in last 5 years C 4 

Research grant (in crores of Rs.) C 5 

Campus area (in acres) C 6 

Placement of UG and PG students (in %) C 7 

Number of books and online Journals available in library C 8 

Course fee (in Rs.) C 9 


MHRD has already recommended to maintain a teacher student ratio of 1:12 (0.083) 
in the NITs. The total number of enrolled students may be another important criterion, 
but as it will have a strong correlation with the teacher student ratio, it is discarded 
from consideration. The total number of international/national conferences/seminars/ 
symposiums organized by the NITs during the span of last 5 years (2008-2012) and 
the total number of papers published in the International Journals of high repute during 
the last 5 years are taken as two other criteria. Research grant is the total amount of 
grants in crores of rupees sanctioned by the MHRD of Government of India to each of 
the NITs during the last 5 years to support conducting research activities. Campus area 
(in acres) is the total area of each NIT within which all its facilities are situated. Place¬ 
ment of UG and PG students (in %) is the percentage of passed out students who had 
been absorbed in different industries/institutions in the last year (2011-2012). The total 
number of books and online Journals available in the libraries of the considered NITs is 
taken into account as another criterion. The last criterion is considered as the course fee 
(in Rs.) for each student required to complete his/her target UG degree. Among these 
nine criteria, all except course fee are beneficial attributes where higher values are 
always required. For course fee, being a non-beneficial criterion, its minimum value is 
desired. Table 3 shows the decision matrix as developed for performance evaluation of 
20 NITs based on nine criteria where the relevant information about the NITs with re¬ 
spect to different criteria are accumulated from various websites (NITs’ own websites, 
www.scopus.com,www.wikipedia.org etc.). The criteria weights are calcu¬ 
lated using Shannon’s entropy method, as given in Table 4. 

Now, this performance appraisal problem of 20 NITs is solved using Visual PRO- 
METHEE which can be easily downloadable from www.promethee-gaia.net/ 
software.html website. It is an interactive PROMETHEE-GAIA software. Fig. 2 
shows the input window of Visual PROMETHEE where the related criteria values for 20 
NITs have already been entered. In this window, the user can choose the number of ac¬ 
tions (alternatives), number of criteria, number of scenarios (for group decision-making 
problem), unit for each criterion, type of the criterion (beneficial or non-beneficial) and 
type of the preference function. Here, the simplest form of the preference function (i.e. 





Performance Evaluation of Indian Technical Institutions Using... 


117 


Table 3 

Decision matrix for performance evaluation of 20 NITs 


SI. Name of NIT 

No. 

c, 

C 2 

c 3 

c 4 

c 5 

c 6 

c 7 

C„ 

c 9 

1. NIT, Agartala 

180 

0.077 

29 

46 

3.16 

333.60 

87.70 

46,678 

234,300 

2. NIT, Allahabad 

148 

0.032 

454 

638 

9.46 

222.00 

68.00 

121147 

199204 

3. NIT, Bhopal 

209 

0.047 

70 

283 

16.10 

650.00 

67.00 

128,163 

191,470 

4. NIT, Calicut 

134 

0.048 

58 

291 

21.16 

420.00 

85.00 

127,000 

276,062 

5. NIT, Durgapur 

150 

0.059 

105 

178 

21.00 

187.00 

79.35 

139,500 

202,068 

6. NIT, Hamirpur 

88 

0.049 

87 

476 

17.97 

320.00 

68.81 

57,705 

276,450 

7. NIT, Jaipur 

85 

0.025 

185 

401 

8.54 

312.00 

79.15 

146,960 

340,000 

8. NIT, Jalandhar 

82 

0.038 

81 

542 

10.27 

154.00 

72.14 

114,801 

197,500 

9. NIT, Jamshedpur 

165 

0.052 

355 

920 

8.85 

350.00 

65.32 

62,658 

214,500 

10. NIT, Kurukshetra 

138 

0.051 

119 

547 

17.64 

330.00 

77.76 

58,009 

189,250 

11. NIT, Nagpur 

218 

0.062 

93 

335 

5.15 

215.00 

58.80 

68,000 

166,140 

12. NIT, Patna 

55 

0.029 

8 

30 

4.21 

60.00 

45.80 

56,100 

166,050 

13. NIT, Raipur 

106 

0.034 

33 

51 

2.96 

100.00 

73.85 

99,580 

159,770 

14. NIT, Rourkela 

132 

0.062 

494 

1314 

15.27 

647.42 

88.00 

66,900 

251,500 

15. NIT, Silchar 

200 

0.167 

39 

60 

12.68 

540.00 

74.13 

67,567 

176,280 

16. NIT, Srinagar 

92 

0.048 

2 

7 

7.57 

297.58 

82.00 

85,850 

168,700 

17. NIT, Surat 

152 

0.076 

222 

721 

22.93 

295.00 

71.28 

78,500 

160,000 

18. NIT, Karnataka 

110 

0.020 

178 

854 

21.87 

250.00 

85.00 

123447 

169,150 

19. NIT, Tiruchirappalli 

246 

0.068 

333 

1667 

20.00 

800.00 

94.60 

134,495 

161,050 

20. NIT, Warangal 

200 

0.103 

247 

527 

14.21 

248.00 

95.00 

175,355 

171,450 


Table 4 

Criteria weights using Shannon’s entropy method 


c, 

c 2 

c 3 

c 4 

c 5 

c 6 

C, 

c s 

c. 

0.1283 

0.1122 

0.0650 

0.0675 

0.1128 

0.1091 

0.1408 

0.1268 

0.1374 


usual criterion) is selected. The user can also select other types of preference function, 
the corresponding threshold values for those functions would then need to be provided. 
After entering all the relevant criteria values for 20 NITs, the corresponding maximum, 
minimum, average and standard deviation values for each of the criteria are automatical¬ 
ly displayed. In Fig. 3, the PROMETHEE II complete ranking of the considered NITs is 
provided, based on the net outranking flow values. In this figure, the top half of the scale 
(in green) corresponds to positive net outranking scores and the bottom half (in red) to 
negative scores. It is clearly observed that alternative 19 (NIT, Tiruchirappalli) tops the 
ranking list, followed by alternatives 20 (NIT, Warangal) and 17 (NIT, Surat). It signifies 
that these three NITs are the best performers with respect to nine evaluation criteria. The 
performance of alternative 12 (NIT, Patna) is not at all satisfactory. It is also interesting 
to observe that alternatives 6, 8 and 13 (NIT, Hamirpur; NIT, Jalandhar and NIT, Raipur) 
also perform unsatisfactorily. The developed GAIA plane is shown in Fig. 4 where the 
positions of the criteria, alternative NITs and n axis are exhibited. The direction of the n 
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Fig. 2. Input window of Visual PROMETHEE for performance evaluation of NITs. 
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PROMETHEE Rankings 
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Fig. 3. PROMETHEE II ranking of 20 NITs. 
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Fig. 4. GAIA plane for performance evaluation of 20 NITs. 





















120 


R. Ranjan, S. Chah'aborty 


axis indicates the superiority of NIT, Tiruchirappalli and NIT, Warangalover the others. 
But in the n axis direction, as the position of NIT, Tiruchirappalli is the farthest from the 
origin of the axis, it is the best choice. A recent report (http: //www. kinindia. 
com/university/top-10-nit-colleges-in-india-2013/) makes a list 
of the top ten NIT performers in the session 2012-2013 as Warangal, Tiruchirapalli, 
Surathkal, Delhi, Allahabad, Jaipur, Calicut, Nagpur, Kurukshetra and Rourkela which 
also matches with the observations from the developed GAIA plane regarding the posi¬ 
tions of the top two performing NITs. It is worthwhile to note here that in the GAIA 
plane, criteria C 4 , C 5 and C g (number of papers published in Journals in the last 5 years, 
research grant, and number of books and online Journals available in library) form a 
cluster. On the other hand, criteria C„ C 2 , C 3 , C 6 and C 7 (faculty strength, teacher student 
ratio, number of conferences held during the last 5 years, campus area, and placement 
of UG and PG students) are in another cluster. The identified criteria in both the clusters 
have the same effects on this evaluation process, expressing similar preferences. As 
these two clusters of criteria are represented by orthogonal axes in the GAIA plane, they 
are also not related to each other with respect to preferences. As criterion C 9 (course fee) 
is a non-beneficial attribute, its position is totally in the opposite direction in the GAIA 
plane, as observed in Fig. 4. NITs 3, 9, 10, 17, 19 and 20 are quite strong with respect to 
criteria C 15 C 2 , C 3 , C 6 and C 7 . On the other hand, the values of criteria C 4 , C 5 and C 8 are 
favorable for NITs 4, 5 and 14. The performances of NITs 4 and 5 (NIT, Calicut and NIT, 
Durgapur) are quite similar in nature. While using PROMETHEE-GAIA method, this 
performance evaluation problem of NITs has a medium quality of solution (reliability) 
of 52.6%, which indicates that it is hard problem to solve. 

The PROMETHEE rainbow diagram for performance evaluation of 20 NITs is ex¬ 
hibited in Fig. 5. In this figure, each bar for each NIT is composed of different slices, 
and the height of each slice is proportional to the contribution of one criterion on the 
final ranking of NITs. Positive (upward) slices correspond to good features while nega¬ 
tive (downward) slices represent the weaknesses. The 20 NITs are positioned from left 
to right according to the PROMETHEE II ranking. From this figure, it is clear that NIT, 
Tiruchirappalli is excellent in performance with respect to all the nine criteria, and crite¬ 
rion Cl (faculty strength) is its most favorable feature, followed by criterion C7 (place¬ 
ment of UG and PG students). On the other hand, the performance of NIT, Warangal 
is quite comparable with that of NIT, Tiruchirappalli, except for criterion C6 (campus 
area). NIT, Warangal has a total campus area of 248 acres as compared to NIT, Tiruchi¬ 
rappalli (800 acres). Criterion C7 (placement of UG and PG students) is the strongest 
feature for NIT, Warangal, followed by criterion C8 (number of library books and online 
Journals). According to the PROMETHEE rainbow diagram, NIT, Patna has the weakest 
performance amongst the 20 NITs and its only good feature is its relatively low course 
fee structure. Fig. 6 exhibits the best to worst ranking of 20 NITs and also the weights 
allocated to nine selected criteria. Clearly in this figure, the considered 20 NITs form 
two separate clusters based on their net outranking flow values. NITs 19, 20, 17, 14, 15, 
18, 5, 3, 4 and 10 are in one cluster having positive flow values, whereas, NITs 9, 11, 1, 
2, 16, 7, 13, 8, 6 and 12 form another group with negative flow scores. It signifies that 
amongst 20 NITs, exactly ten are satisfactory performers and the remaining ten NITs are 
underperformers having deficiency with respect to some of the considered criteria. In 
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Fig. 5. PROMETHEE rainbow for performance evaluation ofNITs. 



Fig. 6. Walking weights for performance evaluation of 20 NITs. 
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this window, by changing the criteria weights (walking weights), the user can also visu¬ 
alize the effects of those changing weights on the subsequent ranking of NITs. It basi¬ 
cally acts as a visual sensitivity analysis tool. In Fig. 7, the visual stability interval of the 
most important criterion C7 (placement of UG and PG students) is provided. It shows 
how the multi-criteria net flow scores change as a function of the weight of criterion C7. 
In this figure, the horizontal dimension corresponds to the weight of criterion C7 and the 
vertical dimension relates to the net flow scores. For each NIT, a line is drawn showing 
the net flow value as a function of the weight of criterion Cl. At the right edge of this fig¬ 
ure, the weight of the criterion is 100% and the NITs are ranked here with respect to cri¬ 
terion Cl. At the left edge, the weight of the criterion is equal to 0%. The position of the 
vertical green and red bar corresponds to the current weight of criterion Cl at 14.08%. 
The intersections of different NIT lines with the vertical bar also provide the complete 
ranking of NITs. It is observed that the weight stability interval of Cl criterion ranges 
between 13.90% and 14.86%, which signifies that within this interval, there would be no 
change in the positions of the top-ranked NITs. It is also quite interesting to observe that 
at the higher weight of Cl criterion, NIT, Warangal supercedes NIT, Tiruchirappalli, and 
NIT, Patna remains as the underperformer along the entire range of Cl criterion weight. 
Similar types of weight stability intervals can also be derived for other non-important 
criteria to search out the possibility of any rank reversal among the considered NITs. 


Visual Stability Intervals 


+ 1 


OT — 


-1 

nz 

0% 


14% 

zc 


X 




.-10 


+20 

19 

14 
1 

48 

16 

5 

7 
10 

15 
13 

8 
17 

6 
2 
3 
9 
11 
112 


25% 


Weight 

0 Stability level: 


75% 


20 : WSI: [ 13.90% , 14.86% ] 


Fig. 7. Weight stability interval of criterion C 7 


















Performance Evaluation of Indian Technical Institutions Using... 


123 


8. Conclusions 

In this paper, an attempt is made to integrate PROMETHEE and GAIA methods to evalu¬ 
ate the comparative performance of 20 NITs in India based on nine uncorrelated criteria. 
In order to avoid subjective judgments, Shannon’s entropy method is used for computing 
the criteria weights. It is observed from the analysis that NIT, Tiruchirappalli is the best 
performer followed by NIT, Warangal. NIT, Warangal is lagging behind NIT, Tiruchi¬ 
rappalli only with respect to its total campus area. NIT, Patna obtains the last position 
in the ranking list among 20 NITs and it has only one favorable feature as its relatively 
low course fee structure. The walking weights analysis helps the user to observe the ef¬ 
fects of changing criteria weights on the final ranking of NITs. On the other hand, the 
user can also determine the weight stability intervals of different criteria to identify the 
possibility of any rank reversal among the considered NITs. This performance evalua¬ 
tion analysis will help the MHRD of Government of India to identify the strengths and 
weaknesses of NITs, and take proper remedial measures to better the positions of the 
underperforming NITs. The performance of other higher education institutions, like IlTs 
and Indian universities can also be evaluated using the adopted approach. 
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Indijos technin| issilavinima teikiancii; instituciji; veiklos vertinimas 
PROMETHEE-GAIA metodu 

Rajeev RANJAN, Shankar CHAKRABORTY 

Siekiant kokybiskesnip mokslinii} tyriimj ir egzistuojancio mokymo praturtinimo, vis svarbes- 
nis tampa techninj issilavinimq teikiancii} instituciji} veiklos vertinimas. Sio vertinimo rezultatai 
galeti} buti atskaitos tasku priimant sprendimus del finansines ir kitos paramos teikimo konkreciai 
institucijai. Instituciji}, atsakingi} uz techninii} specialybii} propagavimq tokioje salyje kaip Indija, 
veikla taip pat reikalauja vertinimo globaliame ekonomikos kontekste. Straipsnyje pristatomas 
integruotas PROMETHEE ir GAIA metodas taikytas 20 Indijos valstybinii} technologijos instituti} 
veiklai vertinti atsizvelgiant j devynis pagrindinius kriterijus. Tyrimo rezultatai parode, kad re- 
miantis pastanyi} penkii} meti} duomenimis, Tiruchirappalli ir Warangal valstybiniai technologijos 
institutai yra efektyviausiai veikiantys. Kita vertus, Patna valstybinis technologijos institutas yra 
paskutineje pozicijoje, todel siam institutui tureti} buti siuloma daugiau isteklii} efektyvesniam 
valdymui ir atitinkamai pletrai. Pagrindinis kriterijus yra atitinkami} instituti} absolvent!} padetis. 



